Various electronic devices, for example, cameras, mobile phones, and other multimedia devices are widely used for capturing image of a scene. Some electronic devices are capable of capturing stereoscopic images, and performing disparity map estimation of the scene using the stereoscopic images. Disparity map estimation is a process that determines shifts in pixels between the stereoscopic images. Oftentimes, the disparity map includes incorrect disparities around object contours and missing disparities represented as holes and textureless regions. Normally, refining and filling of disparities in the disparity map are performed using a variety of methods, such as a minimum spanning tree (MST) based aggregation framework, a horizontal/vertical based aggregation framework or the like. However, refining the object contours and resolving errors in disparity estimation due to holes and textureless regions has been a challenge.